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PROJECT OVERVIEW 


Purpose 

The purpose of this project was to review six of the RSD and DLT web sites to determine their 
“school readiness.” School readiness was defined as the degree to which the site is valuable and 
usable by the K-14 classroom teacher and in what capacity. For each site, a determination was 
made as to whether the site could be recommended to classroom teachers as is. If the site was 
determined not to be school ready, development strategies were recommended to improve its 
educational value. 


Process 

A team of reviewers was selected to review each site first individually and then collaboratively 
with other team members. A web site strategy was selected to facilitate team collaboration. Each 
individual review was posted on the web. Team members were encouraged to visit the site to 
review others’ comments. Face-to-face meetings were then arranged with the team members and 
the Instructional Design coordinator to focus the reviews. Many were rewritten and refined based 
on this process. Final recommendations were then made for each site and are summarized in this 
report. The initial, detailed review of each site is available upon request. 

The Review Team 

The review team consisted of K-12 classroom teachers, university science pre-service teacher 
educators, and university level instructional designers. Each team is described below. 


K-12 Classroom Teacher Team 

Six teachers participated in the review of the materials. The teacher team varied in level of 
experience, curriculum, and technical expertise. Their schools varied in the availability and use 
of technology in the classroom. Together they brought a wide range of classroom experience to 
the project. The teachers represented urban, suburban, and rural areas from two states. More 
details on each of the team members is included in Appendix A. 

The teachers were asked to focus on how usable the site was in the K-12 classroom and how 
appropriate the content and activities were for K-12 students. 

The main criteria for assessing the sites “school readiness” for the teacher team was how quick 
and easy it was to find, comprehend, evaluate, and implement educational materials from the site 
into their curriculum. Access appeared to be the litmus test for teachers investigating web-based 
materials for their classrooms. More specifically, teacher reviews focused on the following areas: 

Meaningful learning activities - were there useful and meaningful learning activities available on 
the site? Were they easy to find and evaluate? Were they based on clearly defined learning 
objectives? Did the learning objectives integrate with district requirements? 

Topic/heading clarity - Could the user identify main point of subtopic heading/hyperlink before 
linking to the sight? 
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Audience - Did the subtopics, information and/or activities clearly identify the audience for the 
material? 

Level of technology - What level of technology was needed to effectively use this site? Was the 
site adaptable to the available technology? What specific hardware and software were needed to 
optimize use of the site. 

Implementation of activity or information - Did files need to be downloaded? Were they 
compatible with available hardware/software? Was it time-consuming? What were the materials 
and resources needed? Were there any browser configurations and/or add-ins required to use any 
of the resources? Were other applications needed to run downloaded activities? 

Other specific questions - Did the site offer information that was not easily accessible in other 
readily available forms? Did it introduce a type of interaction that was difficult to do in 
classroom but easily accessible through web site? Did it include innovative teaching and learning 
strategies that the user can implement? Were there applications for teachers of other disciplines 
and were these interdisciplinary activities easy to find and evaluate? 


University Science Pre-Service Teacher Educators 

A two-person university science pre-service teacher educator team brought a broad perspective of 
science and education to the review. These educators were able to provide a point of view based 
on the most current advances in science and math education theory. One team member was from 
the science education program and the second was from the math education program of 
Curriculum and Instruction in the College of Education at the Pennsylvania State University. 
Their backgrounds are described in Appendix A. 


University Level Instructional Designers 

Three team members represented the instructional systems design viewpoint, although one 
member functioned as the primary reviewer. Two were university professors, and one was a 
student in the program. The second member on the team functioned also as the Project 
Coordinator, and the third as Principal Investigator. Both of these team members were able to 
bring together the points of view of the classroom teacher team, university science pre-service 
teacher educator team and the other instructional design perspective from an overall instructional 
systems perspective. Their backgrounds are described in Appendix A. 

The instructional systems design view pulled together instructional theory, learning theory. The 
reviewers evaluated the design of the site in terms of the stated goals and the use of web-based 
technology. Their investigation was based on the degree to which the instruction matched stated 
learning objectives, used appropriate instructional strategies to support the learning objectives, 
contained appropriate content for the learner and supported the learning objectives, and had 
appropriate media to support learning and made the best use of that media. 
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REVIEWS 


SAIC - ATHENA: CURRICULUM DEVELOPMENT, IMPLEMENTATION, & 

SUPPORT ON THE INTERNET 

Summary Comments and Recommendations 


K-12 Classroom Teacher Perspective 

Most teachers felt the Athena site had a nice look and feel. The design could be improved to 
make it easier for teachers looking for meaningful learning activities and enhancement to their 
own curriculum. Links could lead a teacher to wide ranging levels of materials. Some areas 
included links to activities that involved little more than “page flipping” on the part of the teacher 
and/or student. There was inconsistency among the component areas in terms of organization, 
level, and type of content/activities. Because of this inconsistency, a summary for each content 
area of Athena is included. 

1. Space - Space is the most sophisticated and best organized of the Athena sites. Activities 
vary in level from primary to high school level although this is not clearly indicated with the 
lessons. (A separate page - “Directory for Teachers” contains this information.) A separate 
link to “Teacher Talk” sections is helpful but is available for only a few of the lessons. All 
the teachers appreciated the fact that this portion of Athena did not have a regional bias. The 
site has a good general overview that sets the framework for further investigation. It is brief 
enough to use in class and does not contain links that could lead students away to other Web 
sites and possibly cause confusion. One teacher felt the overview was at too high a level in 
terms of content and language. Teachers recommended the use of some simple analogies to 
make some of the concepts more understandable. 

2. Weather - In general, this site was aimed at the lower grade levels. The teacher panel felt 
that most activities were appropriate for K-5 and middle school students. 

3. Oceans - The Ocean site has some very interesting sensing data available (drifter buoys, for 
example) but in many cases misses opportunities to expand on the information. 

4. Earth - Although showing its West Coast (and Washington State especially) bias, parts of 
this site provide good opportunities for interesting student activities. Some of the links 
provided richer information and better student activities than the site itself. 

5. Teacher Materials - There are various teacher-related materials available on the site. 

• Especially for Teachers provides some background on the Athena project and some pilot 
teacher reflections. Background of the project was of interest to teachers. 

• Directories for Teachers provide general guidelines in the form of a grid that provides 
grade level information for each unit and activity information. The grid clearly shows 
one of the problems with the site - the preponderance of information vs. meaningful 
activities. 

• Teacher Talk provides guides to lesson objectives, management tips, vocabulary, links to 
standards as well as to additional resources. This material is difficult for teachers to 
work with due to some lack of consistency (for example compare Teacher Talk for 
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“Observing Comet Hale-Bopp” vs. “Class Tour of the Solar System. Some provide links 
to Washington State standards (Essential Academic Learning Requirements) which are 
not that useful to other teachers around the country. National Standards would be a better 
choice. The activities link to the EARL standard and are described but do NOT link to 
the activity in the site itself. Also, many teachers are looking for specific activities and 
enrichment in the site itself and will bypass this material altogether. 

Based on the teacher review, the following recommendations are made regarding this site: 

1 . Teacher Talk needs to be expanded to all of the lessons in the site. It is difficult for teachers 
to evaluate the appropriateness of lessons and activities without clearly stated learning 
objectives. There are some links to Washington State Standards on some of the Teacher Talk 
pages. National Standards would be a better choice. In addition, there is no link from an 
activity based on a standard to the actual activity in the lesson. 

2. The site often makes use of Macintosh only and other activities require a fairly powerful 
computer with a fast Internet connection. A guide to the technology necessary would be 
helpful. There is a page that may do this but is now “under construction”. Also a clear 
statement of what software, browser plug-in and browser version needed to download and use 
the activities is needed. 

3. Make the site user-friendlier for teachers trying to find specific activities and enhancements. 
Most teachers across the country will likely not use whole lessons from this site but review it 
for additional resources for their own curriculum enhancement. Some type of teacher 
organizer that goes well beyond “Directory for Teachers” is needed to help them identify and 
evaluate materials appropriate for their classrooms. 

4. There are aspects of the site that may be used in ways other than the designers intended. For 
example, some of the material from the lesson on orbits could be refined and used in a 
Geometry class. Also, teachers could develop much of the on-site data as well as linked-data 
sites into lab-type activities for their own classrooms. 


University Science Pre-Service Teacher Educator Perspective 

The Athena homepage is nicely designed and is colorful and compact with appropriate links. 
However, the site lacks activities for students. More conceptual material should be included 
rather than having a majority of the site being words and definitions, which focus on “naming 
things.” More visuals are needed that explain and elaborate on the concepts. New visuals should 
replace existing ones. 

Guidance for teachers was “site management” oriented and did not address goals for learning. 
More recommendations are needed that suggest important ways of using the site, what learning 
goals can be achieved, and what the students can learn by using this site. 

There was a lack of consistency in the site designs. For example, the Weather site included grade 
level recommendations, others include this only in separate Directory for Teachers pages. 
Learning goals are included only in a few Teacher Talk pages. All material needs to be related to 
learning goals. National Science Standards could be used as a framework. Provide more 
connections for teachers and students with additional references that go well beyond the 
superficial and clearly relate to learning goals and activities. 
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From a math intensive perspective, the Scavenger Hunt is the most ready for students, and Using 
Spreadsheets is the least school ready. The Time and Scales sections are inappropriate for those 
teachers/schools who have adopted the 1989 NCTM Standards mentioned above. The methods 
these sections currently use do not encourage students’ conceptual understanding but focus 
merely on their procedural knowledge. The General Information and Activities section has 
potential for integrating mathematics in the curriculum. 

Based on the university science pre-service teacher educator review, the following 
recommendations are made regarding this site: 

1. The site has a lot of potential for becoming school-ready. 

2. The site could use a site-wide external learning framework as an organizer for teachers. 
Perhaps a combination of “Teacher Talk” and the "Directory for Teachers” could be a start. 
This framework needs to be consistent and applied to all of the Athens lessons. Specific 
objectives need to be stated for each lesson as well as ties to standards that all teachers can 
relate to i.e. National Standards. 

3. The site needs to steer away from its focus on words and “naming things” to meaningful 
learning activities based on objectives. 

4. Links need to be reviewed for appropriateness for the students and labeled as such. 

5. Many opportunities for connections both within the site and to external sources need to be 
made. 

6. Find retired scientists and science educators to provide “expert resource” for students and 
teachers. They are more apt to contribute than working scientists. However it is still 
necessary to develop a system to filter communications to these experts so they are not 
overwhelmed. 


University Level Instructional Design Perspective 

From the ISD perspective the site has two major problems - the absence of clear purpose or utility 
for the information in the sites and the nature of the student interactions/activities. The Athena 
site suggests a number of activities, but provides very little support for those activities. 
Additionally, the activities contain very little scientific substance, that is, did not support any 
meaningful scientific activity. 

The Athena site contains a lot of information that can be read and memorized if assigned by the 
teacher. The organization is subject matter related, just like a textbook. In essence, what the site 
provides is an electronic textbook of sorts. Students have the opportunity to select what they 
want to read or view next. 

However, what is needed is a purpose for using the information contained in the site. Students 
need to be given a purpose for accessing the information. It may be that some students have an 


Final Evaluation Report 


5 



intrinsic interest in the information. More than likely, accessing the information will be to fulfill 
the requirements stated by the teacher. 


Based on the instructional design review, the following recommendations are made regarding this 

site: 

1 . Focus content according to larger problems or projects, and within projects, according to 
activity structures that support specific learning objectives. Add a driving question - a 
problem, a conundrum, a puzzlement, a discrepancy - some activity-based purpose for 
accessing information as a resource to motivate students to really dog into the information. 
Having identified that problem, the activity needs to be analyzed in terms of how the site can 
support solving the problem. 

2. Add problem-solving activities, real scientific investigation, and assessment aimed at how 
students would know if their investigations were successful. Rather than having students 
write poems or act out plays about the weather, have them function like meteorologists, 
making predictions that might save lives or crops. 

3. Provide specific assessment rubrics, replete with evaluation criteria, checklists, etc. for Web 
developers. 

4. Expand interactivity in this site. 

5. Include practice activities for students in support of clearly stated learning objectives. 

Expand use of graphics in support of concept learning. 
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SPACE AVAILABLE: AQUATIC APPLICATIONS OF SATELLITE IMAGERY 
Summary Comments and Recommendations 


K-12 Classroom Teacher Perspective 

In general, the teacher team felt the Space Available web site was attractive, colorful, and full of 
interesting information and activities. Most were frustrated, however, by a lack of guidance for 
the teacher to access and find materials appropriate for their classrooms. The teachers felt the site 
was a good resource for general information and some activities but that it took too long for them 
to find materials that might be useful. ’Teacher" pages need to be added to assist the instructor in 
the use of the web site and its application to the curriculum. In addition, a "teacher guide” or 
lesson plan outline would be useful as a quick reference as to what each link contains. Although it 
is an excellent source of information and a wealth of activities, the lesson ideas could be focused 
on more meaningful learning activities. As it stands now, the information is pertinent and useful, 
but classroom teachers would need too much additional planning time to decide how to use the 
web site with students. 

An interesting feature in this site is the use of NIH Image, a public domain image processing and 
analysis software package. NIH could develop to be one of the more important tools available on 
the Internet. However it will require sufficient computers to allow students to interact or at least a 
computer interfaced to a projection device to be of any use in most classrooms. One of the 
cautions associated with NIH is that it will require a computer sufficient ram and a high- 
resolution color monitor to be effective as a classroom tool. Another caution is that, at present, 
this software is available only for Macintosh computers. 

This site is also one of the few to attempt to integrate interdisciplinary activities. Lessons are 
presented using math, reading, as well as writing skills. 

Including long term, open-ended projects is recommended to the site designers. The site presents 
many opportunities to chart data and variables over a long period of time as well as the chance for 
groups of students to interact with each other either locally or virtually over the Internet so that 
findings can be compared. 

Summary comments are provided for each of the components of the site. 

Comparing Oceans . Although focused on the Maine environment, the content and activities are 
applicable to students in other regions. Most sections of this portion of the site come with 
developed lessons and/or suggestions for meaningful activities. The majority of these activities 
can be completed with out having to be on-line. The most positive comment is that teachers can 
use this site as a source for developing interesting classroom activities. 

Human Impact. Most of these activities were interesting but tended to be shallower in terms of 
content. Perhaps the intent was to aim this section at lower grade levels only, but this is not 
spelled out. For this section of the site to be more effective in high school, there would need to be 
considerable development and more meaningful threads. 

Antarctica. There are some very interesting and cursory activities here. However, in some 
instances, the lessons are so limited in content as to actually convey errors in science. This section 
contains a lot of interdisciplinary skills but more limited science content. 
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Weather. There are many interesting concepts in this section of the site but they could be better 
supported. Unless properly introduced by the teacher, many activities would seem to be nothing 
more than '’busy work.” This section of the site doesn't seem as well organized as the others. 

Imaging the Earth . This is a very informative and instructive unit comprised both of reading and 
action activities. A real plus is the fact that many activities can be used in rooms with limited 
Internet access. There are very interesting activities involving map constructions and projections 
as well as opportunities for involving students in cooperative learning. 

Remote Sensing. This site provides a good introduction for elementary and middle school 
students on this topic. With some enrichment of the material, this could be an excellent resource 
for upper grades as well. 

Based on the teacher review, the following recommendations are made regarding this site: 

1 . Aim activity descriptions at student with separate instruction pages for teachers. 

2. Provide an overview for teachers (perhaps a "teacher map" with more complete explanations 
of the links and activities to be found. 

3. Spell out specifics for teachers so that it is easy to determine objectives and standards 
supported by the activities. 

4. Include specific learning objectives and lesson plans to support activities. Add more content 
enrichment sources and links for teachers. 

5. Provide some more complex problem solving scenarios to help make activities more 
meaningful for students. 

6. Develop projects that require students to access, chart and manipulate data over time. 

7. Provide approximate time teachers should allocate for activities to be completed by students. 
Include materials list in "teacher pages" with an overview of the activity. 


University Science Pre-Service Teacher Educator Perspective 

The Gulf of Maine Aquarium homepage is attractive and colorful. The amount of material 
directed to teachers to assist them in preparing their lessons appears to be minimal and is 
frequently contained in information that was directed toward students. There is a wealth of 
material and activities under this topic however, they are embedded too deeply and their titles do 
not clearly convey what the topics and/or activities are. Feedback was not (frequently) requested. 
There seem to be few links that incorporate any mathematics or mathematics activities. 

In order to incorporate more mathematics in this section especially for students in junior or senior 
high school, include another activity in which mathematical modeling is used to estimate the 
number of some type of fish in this body of water. Include more pictures where possible. 

Space Available provides many interesting activities for students but could really do much more 
to provide rich science content for both students and teachers. Topics like the "greenhouse 
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effect" need much deeper content presentation to get across the correct concepts to students. 

"Save the Bay" is another example of a topic that does not have enough background resources to 
make the activities more meaningful to students. For this site to be a better resource for teachers 
there needs to be much richer science content directly supporting the activities. Some of this 
enriched content could introduce the activity for students; could be added to separate resource 
pages for teachers, and/or available in "linked resources" for both students and teachers. Perhaps 
references to national standards and to major textbooks used in kl2 would help teachers integrate 
into the curriculum. 

Based on the university science pre-service teacher educator review, the following 
recommendations are made regarding this site: 

1. Increase the amount of material directed to teachers to assist them in preparing their lessons, 
and disembed them from material for students. 

2. Provide a guide that clearly conveys what the topics and/or activities are. 

3. Increase the number of links that incorporate mathematics or mathematics activities. In order 
to incorporate more mathematics in this section especially for students in junior or senior 
high school, include another activity in which mathematical modeling is used to estimate the 
number of some type of fish in this body of water. 

4. Include more pictures where possible. 

5. Increase the richness of science content for both students and teachers. Some of this enriched 
content could introduce the activity for students; could be added to separate resource pages 
for teachers, and/or available in "linked resources" for both students and teachers. 

6. Add references to national standards and to major textbooks used in k 1 2 would help teachers 
integrate into the curriculum. 


University Level Instructional Design Perspective 

Space Available is divided between interesting information bits and instructional activities. The 
information will be useful to someone who is interested. If they are not interested, memorizing 
these bits will be like any other academic chore. The instructional activities, for the most part, are 
traditional, prescriptive activities that support only surface level thinking. There are some good 
comprehension goals but these issues and ideas are relatively disconnected. There are, however, 
many interesting problems in the Gulf of Maine. The site could do a better job of using the web 
to provide tools and resources to engage students in solving those problems. 

The satellite imagery provides a powerful kind of visualization tool, i.e. tools for visualizing 
phenomena. They show important relationships and could be valuable tools, when provided a 
reason to use them. The satellite images could be better used to help students to analyze and solve 
problems. 

Based on the instructional design review, the following recommendations are made regarding this 
site: 

1 . Add a strong, relevant learning purpose to the site. 
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2. Make the activities more powerful by having them explore what human adaptation and 
technological creativity have been used to deal with the issues of the site. 

3. Develop activities to support interesting questions like: whose fault it is, who owns the fish, 
how the problem is like other ownership problems, how the US and Canada might negotiate a 
fairer deal, etc. 


Final Evaluation Report 


10 



PRIME TECHNOLOGIES: URBAN ENVIRONMENT INITIATIVE 


Summary Comments and Recommendations 


K-12 Classroom Teacher Perspective 

This site provides a wealth of information about the environmental, economic, and social factors 
that combine to create the urban world in which many of us live. The information and ideas 
behind the site are insightful and provocative. It pulls together the past, present, and future of our 
society. As the site presently exists, it is primarily a resource of information along with 
interesting mapping functions for analysis. The content areas could easily connect with Science, 
Social Studies, Math, and Language Arts. However, most of the teachers felt the site would 
require quite a bit of effort to integrate the material into the curriculum although it could certainly 
be used as a research resource for students. The site could be much more useful in K-12 
education with the addition of teacher resources and the development of meaningful learning 
activities for students. This site can provoke ideas for the development of learning tools for 
studying the social, economic, and environment of student’s local community. The information 
provided here encourages a more broad, holistic view of their community in relation to the 
environment. 

The teachers were very positive about this site and noted the following specific strengths of the 
site: 


• Subject matter - The role of natural environment indicators of change is extremely important 
for identifying long term consequences of development that can affect social policy decisions. 
This site can serve as an example of how "science, technology, and society" are related to the 
overall use of land resources. There is much textual reference material here along with the 
opportunities to analyze data graphically. There is potential for teachers to develop lessons in 
math, science, environmental education, economics and social studies. 

• Use of satellite technology - The great IR photography is presented here in a form that could 
be useful to the communities where students live. Students can learn how urban planners, in 
forecasting problems associated with rapidly growing population centers, can utilize a 
modem technology such as remote sensing. 

• Varied method of analysis - The process of comparing maps with common parameters is an 
excellent and valid systems analysis tool. A variety of queries and the different levels of 
view, regional and community, are practices that can be valuable to students when 
determining the scope and depth of an issue. 

• Limits of remote sensing - The map interpretation material includes comments on the 
limitations of representing this data in graphic form. To consider these factors in a social or 
environmental setting is very important. 

Based on the teacher review, the following recommendations are made regarding this site: 

1. Guides should be provided that include resources and meaningful learning activities for 

students to help teachers integrate the material into the curriculum; although it could certainly 
be used as a research resource for students. 
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2. Teachers could certainly use many of the questions explored in the site to develop interesting 
lessons and activities for students. 

3. Research Tool -This site provides a wealth of information. For teachers and students, the 
current site would aid in research in many interest areas: The three regions; Air pollution; 
Perceptions and Assessment of Global Environment Change; Social Dimensions of Resource 
use; Impacts of Structures and Institutions; Industrial Growth; Hazard Waste; Human 
Ecology. 

4. Students can explore their own local communities through the questions posed by the site. 
Teachers can develop work sheets using these questions to develop projects for students to 
more fully understand their communities. 


University Science Pre-Service Teacher Educator Perspective 

The site presents a lot of information about an important topic for analysis but unfortunately, 

provides little support for students and teachers. As is, it can be used for student research. Some 

of the questions posed can be used by students and teachers to analyze their own communities. 

Based on the university science pre-service teacher educator review, the following 

recommendations are made regarding this site: 

1 . Redesign for fewer words on a page. It would be more effective to add charts and other 
graphics to break up the text. Make the text available as downloadable files (i.e. pdf format) 
so teachers can print them for students. 

2. Present complex data, numbers and percentages in charts and graphs so that students can 
more easily understand the numbers. 

3. Redesign the site for increased student interaction. 

4. With the addition of a "teacher map,” lesson plans, and links to background material, this site 
could have more educational value. 

5. Add activities that require students to use all of the data available on the site. 


University Level Instructional Design Perspective 

This has the potential to be a really great educational site. Getting students to face problems 
plaguing urban environments is important. This site can show students how planners analyze all 
the issues and data to make important decisions affecting the future of a community. The site 
provides a wealth of resources, both textual information and data, and means to visualize the data. 
Unfortunately, what it doesn’t provide is the glue to hold those resources together. That "glue" 
can take numerous forms. First and foremost, it needs a purpose, which should be provided by a 
driving question, a problem, a project, some kind of goal structure for assimilating. What is 
needed is a problem for which the information that is available in this web site might be used. To 
get students to develop meaning from all of the information, provide a compelling, driving 
problem that will motivate them. 
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Based on the instructional design review, the following recommendations are made regarding this 

site: 

1. Reorganize and use hyperlinks to more closely tie information to the questions and perhaps 
some more complex problems. 

2. Thematic Guides - The thematic guides do not really represent themes. They provide 
information and some data on socioeconomic structures, racial mixes, air quality, land use 
etc. For educational purposes a theme like "Equity vs. Exploitation" would provide better 
focus for students to use and analyze all of the information provided. 

3. Image Gallery - includes population, crime statistics, toxic sites, vegetation, etc. They are 
interesting images that students would like. As is, only incidental learning would take place. 
Students need reasons to use this material. For example, tasks like " write up legislation that 
could be introduced to the Los Angeles City Council that would provide incentives for 
businesses that would result in reductions in economic inequities among certain 
neighborhoods ” gives students a reason to analyze and use the information and graphics 
available here. 
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UNIVERSITY OF NORTH DAKOTA - VOLCANO WORLD 


Summary Comments and Recommendations 


K-12 Classroom Teacher Perspective 

Volcano World is an information rich site, appealing to a multitude of age ranges. An excellent 
resource for teachers, it gives an abundance of information in accessible links. A rich and diverse 
resource such as Volcano World is much appreciated by teachers. It compiles diverse resources 
into one site, which fulfills a task that is laborious if using books. The site designers have used the 
power of the WWW to collect not only textual information, but graphics, movies and sounds as 
well. This allows for a plethora of student research options that are conducive to different 
learning styles. The site allows teachers to develop lessons that would allow students many 
options for developing projects - newspaper articles, ’Virtual" trips to volcanoes, model building, 
etc, using the various resources included in this site. Some of the success of this site can be 
attributed to the very focused topic. Other NASA sites with more global topics tend to offer more 
disconnected information. In Volcano World, a specific topic is explored more thoroughly. 

The teachers were also very positive about this site and noted the following specific strengths of 
the site: 

• Timeliness - It is obvious that the site is constantly being updated and maintained. There is 
always the most current information available and even volcano eruption alerts sent by email 
to subscribers! Dead links do not appear to exist that are an unfortunately part of many other 
sites. 

• Search Engines - An innovative and very useful tool for quickly finding resources of interest 
in this very large and diverse site. 

• " Ask a Volcanologist" provides access to real experts. Archives of common questions are 
useful and serve to limit the number of questions asked of the experts - a very good idea. 

This is a excellent way to provide access to experts while at the same time assuring that the 
"volunteer" experts will not be inundated with the same questions over and over again. 

• Glossary - Always a good resource for students, this glossary is well done. Students can 
click on key terms and go directly to the glossary. 

• Teaching and Learning section provides some lesson plans and activity ideas for teachers. 
The lessons provided by Scott Johnson were quite thorough. As with most of the site, this 
section is constantly being expanded and updated. 

• Volcano World Online provides a very complete series of lessons that use online resources. 
Schedules, time requirements, group activities etc. are provided 

• Graphics - A wealth of pictures and movies (both QuickTime and mpeg formats) are 
available for those with fast Internet connections. "Dante’s Peak" link interests students, 
many of whom have seen this movie. Movie reviews by real volcanologists provide students 
with an understanding that some "dramatic license" is used in the movie and some 
occurrences are not based on scientific fact. 
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Kid's Door is a fun link especially for young children providing lots of art, contests, school 
web pages, and stories. 




Based on the teacher review, the following recommendations are made regarding this site: 

1. The teacher materials and resources could be better organized, perhaps by grade level. Link 
to activities in the “Kid's Door.” 

2. Lesson plans need more consistency in format, completeness, and quality. The lesson plans 
created by Scott Johnson (for grades 5-8) could be used as a guide. Specific learning 
objectives/outcomes should be included with the lessons. 

3. Teachers should be warned of the long download times required of the graphics so that they 
can be downloaded prior to class. The site is very busy, perhaps some mirror sites would 
help speed things up. Perhaps the "heavy" graphics and basic site organization could be put 
on CD-ROM and the web site could be accessed for updates and the latest information. 

4. Many activities could be improved and made more meaningful with the addition of study 
guides for students. These could be included in the teacher section to be downloaded and 
adapted by teachers for their particular classroom. 

5. Kid’s Door - Rocky’s Adventures could be improved by linking it to the teacher material more 
closely. Many of the activities here could be improved with the addition of better study 
guides. On-line quizzes might be a nice addition to add interaction and reinforcement of 
learning concepts. 


University Science Pre-Service Teacher Educator Perspective 

Volcano World is well-designed site with a wealth of information, graphics, movies, activities 
and resource materials about volcanoes. The variety of data, news, lessons, graphics, movies, 
resources is certainly a strength. 

Strengths of the site include its organization and navigation, currency that provides the latest 
information on currently active volcanoes, glossary, links to experts that includes FAQ files that 
help shape the quality of questions that students ask of the expert. The site also includes a human 
side with its background information on the experts. 

Based on the university science pre-service teacher educator review, the following 
recommendations are made regarding this site: 

1. Providing different levels of sophistication could enhance the resource materials. 

2. Links to outside reference materials could be improved by providing more links to textbooks 
and curriculum materials used in schools. 

3. Provide more suggestions for how to use the site resources in the classroom. 
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4. Provide better links from lessons to Volcano World information and activities. 

5. Graphics take much too long to download. Provide some guidelines for teachers so that they 
can download detailed images outside of class time. 


University Level Instructional Design Perspective 

This site provides information on all active and inactive volcanoes, including recent reports, 
images, background information, pictorial tours of volcanic parks, volcanoes of other worlds with 
lots of video clips of volcanoes. Each volcano description includes pictures (and sometimes 
video), data, history, and some information about the volcanoes. Volcano World is a carefully 
crafted and well-organized collection of information about volcanoes. It could constitute a 
valuable instructional resource if students entered the site with specific questions or purposes. 

This site requires that users have a suite of tools and players and plug-ins. However, since most 
are available as shareware or freeware, this should not present too much of a problem. 

There are several good navigational options. Hot links are embedded in text, provided in on- 
screen menus as well as a pop-up remote control, and a menu bar at the bottom of the screen. 
Navigation is easy. The learning the structure of the program is relatively straightforward and can 
be mastered in a half-hour. 

The web site provides Volcano Lessons, Lesson Plans, which includes teachers guides (text about 
volcanoes), tests, and review sheets, vocabulary sheets, multimedia stacks, movies, and a variety 
of other multimedia resources to drill students about volcanoes. 

Based on the instructional design review, the following recommendations are made regarding this 
site: 

1. The educational materials at the site describe what experts know about volcanoes. The 
questions that the developers should ask themselves are: 

• What do students want to know about volcanoes? 

• What might they want to learn? 

• Why would they want to learn it? 

• What tasks or problems could we give them to get them interested? 

2. Develop some meaningful learning objectives and problems for students to solve using the 
site as a resource. 

3. It would be easy to hyperlink some of the existing lessons to material in the site (especially 
the graphics and movies). Future lessons should be developed with this in mind. 

4. The Volcano simulator in the Kid’s section is the best part of the site. Integrate that with the 
lessons. Have students predict the effects of a volcano in their hometown by using the 
simulator and demographic information available elsewhere on the web. Perhaps this could 
be accomplished using resources available at another NASA sponsored site - "Urban 
Environment Initiative”. 
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5. The Frequently Asked Questions section could be very useful. The questions need organized 
in a more coherent way. Perhaps an "Ask System" architecture could be developed to support 
solutions of a meaningful problems. 

In summary, there are many very useful elements to the Volcano web site. It is obvious that a 
great deal of effort has gone into its development. It could become something great. That would 
require improving the Learning section by thinking about some kind of problem that would be 
useful and structure the resources and the functionality of the site to support solution of those 
activities. 
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UNIVERSITY OF WASHINGTON: LIVE FROM EARTH AND MARS 


Summary Comments and Recommendations 


K-12 Classroom Teacher and University Science Pre-Service Teacher Educator Perspective 

The review team felt that "Live from Earth and Mars" could provide teachers with a wealth of 
interesting scientific information about Earth and Mars. However, the current lack of focus, 
disconnected ideas, and rather poor use and integration of existing information tend to inhibit 
somewhat its potential effectiveness. Because of the site's title, the review team was expecting 
more connections between the study of Mars and the Earth. Many wondered: What is the 
relationship between the two and why have the developers chosen to present Earth and Mars 
together? Where is the synthesis? What conclusion are we to draw from this information? 
Because of the theme implied in the title of the site, many of our teacher group would not have 
turned to this site for environmental information and activities. 

Most of the site is designed to provide technical data and information about the Mars mission, 
interpretation and presentation of data, and the opportunity to interact with some of the 
researchers. The Weather page and Curriculum page, however, seem to have been developed 
quite differently. Compared to the depth and quality of the rest of the site, these sections seem 
more limited. This site could use more structure and “parallel format” and development 
throughout. 

However, there are many good resources available here and many of "Live’s" attributes should be 
studied for future site development as follows. 

• The Mars Science and Engineering link is probably most constructive and one that any 
teacher of science could use to find some substance to improve their instruction. It is also an 
excellent site to send students to learn how to access excellent science material relevant to 
Mars as well as learn key science information in general. 

• The Pathfinder link might overwhelm most high school students but is a site that would be 
very useful to technical students in electronic science or robotic classes and as a resource for 
teachers. 

• There is a clear, well-written description of the project, providing an important mission- 
statement type introduction. 

• The "create your own weather data plots" is an excellent interactive activity, even though 
focused only on the state of Washington, it still provides students with interaction and 
manipulation and interpretation of data. 

• "Way Cool Tools for Mars Exploration" is an excellent site with lots of activity links for 
younger students. As with the other activity-based links here, they present a wide variety of 
student learning experiences. There is a nice mixture of both independent (individual) 
learning activities and group type interactive experiences. 

Based on the teacher and university science pre-service teacher educator review, the following 
recommendations are made regarding this site: 
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1 . Create more focus to the site. 


2. Connect ideas and integrate existing information. 

3. Create a clearer picture of what the connection is between Earth and Mars. 


University Level Instructional Design Perspective 

The biggest question about Live from Earth and Mars is the connection that learners are supposed 
to make between them. The nature of the information provided about each was quite different. 
The Earth portion provided a melange of meteorological data and images. The Mars 
portion provided some of this, but lots of other unrelated information as well. The link structure 
between the two parts was drastically different and both could be better organized. Although 
there is interesting material here, this site needs to be restructured more purposively. The 
developers should focus on the goal or need are they trying to fulfill. Once that has been 
identified, then different access structures or different scenarios can be designed to support that 
purpose. 

Based on the instructional design review, the following recommendations are made regarding 
specific sections of this site: 

Danger in the Air ! - There are some very good activities here. There are opportunities for 
students to do actual investigations, input data, gather, analyze, synthesize, and interpret 
information. Teacher could pose an opening question, “Why did school closed in Mexico for a 
month?” Excellent question for a teacher to ask to stimulate class discussion and interest, or if a 
student is on line. 

Danger in the Air! Activity # 1 : Does Air Pollution Effect You? - Provides interaction and use of 
the Internet. This activity should have an introductory statement explaining who is going to 
review the information entered, purpose of entering information, and expected response time. 

Danger in the Air ! Activity #2: Effects of Air Pollution on Plant Growth - This hands-on activity 
could be added to any environmental study. Teachers seek out this type of activity to include. 
This experiment could take course over a month, allowing students to experience long-term 
experiments. It is important to specify what kinds of plants work best. For example, Mung 
Beans grow fast which is good for expediency, but they die within two weeks, so the experiment 
could be invalid. 

Danger in the Air ! Activity # 4 : Finding and Analyzing Air Quality Data - Correlating levels of 
pollution and using the internet to retrieve the information could be used in both a math and 
science lesson. The activity provides an opportunity for group work, and use of the internet. If a 
teacher did not have access to the Internet in the class, data could be easily printed out. The 
Mexico City air pollution problems (levels exceeded healthy levels for 312 days in the year 
1988!) can really grab students interest. In order to help them understand the data; disclose the 
pollution levels acceptable and unacceptable pollution levels. When students gather information 
they have some knowledge of what is acceptable or unacceptable, or possible consequences of the 
levels they are viewing. 

Digging in the Dirt! - This module is what most of the team expected from this site. It is a direct 
comparison between the earth and mars using soil. The lab activities presented here are 
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appropriate for junior high/ high school level earth science classes. They introduce the concept of 
"controls" and "variables". Also the opportunity for students together data and interpret results. 
These hands on experiments, data sheets, gathering of data and inference demand the student to 
use higher order thinking. Teachers could use this site for student comparisons between the 
familiar earth to the abstract, mars. If access to the internet is available, students could conduct 
mini projects on the soil temperatures of mars, or a teacher could print out information and use it 
as a class lesson. The hands-on experiments and explanation, and data sheet are excellent for 
recording, and summarizing findings. 

Digging in the Dirt! Mars/Earth Comparison - Data is very interesting and could be used in a 
self-directed study if a student was disciplined to laboriously sift through the data explanations. 
Make the material more accessible to other grade levels. Perhaps provide an additional link for 
that simplifies data so that a 4-8 grader could read. Even most 12-grade students would need 
guidance on reading the chart alone. 

Digging in the Dirt! Mars Soil Data - The complexity of this link contrasts with the rest of the 
lessons that seemed to be geared toward 4-6. A teachers key would be helpful, and also a teacher 
explanation page for aiding in the reading of the Martian soil samples and graphs. The 
comparing of data through graphs, and experimentation is the type of activities that should be 
provided. 

Mission to Mars " Choosing a Landing Site ” - Allows for interaction as students submit 
information. Describe who receives the information, explain what will be done with it, and if a 
response will be given. 

Mission to Mars "Click here to Simulate a Flight to Mars" - Displays close up pictures of Mars. 
Can link to features, which could provide a great data collecting, but the terms are unfamiliar. 
Link to definitions, examples of comparing Earth features to those Mars features. Include a 
glossary link for words. 

Water: A Never Ending Story - This page includes a great diagram of the water cycle. The lab 
activities presented here are well introduced, clearly written and easy to follow. 

Several other teacher activities could be constructed from just using this link. This is an excellent 
resource for teachers to develop lab activities. 

Pathfinder Mission to Mars - List and develop pathfinder as an educational link from the 
Education File that the modules where listed under to Education and Outreach of Pathfinder. 

This material can be easily missed as it is now. 

Pathfinder Mission to Mars - How is Mars Pathfinder Navigation Like a Game of Golf? - Good 
analogy to use in upper level high school, or a school well versed in golf. Have a diagram or 
simulation, with labels to give the analogy a clearer meaning. 
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ESSC: EARTH SYSTEM SCIENCE COMMUNITY 


Summary Comments and Recommendations 


K-12 Classroom Teacher and University Science Pre-Service Teacher Educator Perspective 

The Earth Systems Science Community site is an interactive curriculum for students in 9-12 
grades. Currently, the information is geared toward those formally involved in the ESSC 
program, although links could be used to embellish any environmental curriculum or replace 
aspects of an existing environmental unit. Due to the format of the site, little information would 
be easily accessible for elementary and middle school students. 

In order for a teacher and students to use the site, every student must have Internet access, and 
have a high level of computer/intemet competency. All information for completing the activities 
is found throughout the web, along with the culminating assignment: publishing a student report 
through the site. This limits the audience, although it will achieve the purpose of the curriculum. 

There are great advantages, as mentioned in the philosophy for the site: student interaction with 
scientists; cross-disciplinary approach to learning; authentic tasks - publishing on the Internet; 
and cooperative learning. If a high school were moving toward unit based thematic instruction, 
ESSC could be a solid base. That is operating under the assumption more support would be given 
if one became part of the program. 

A possible weakness of the program could be individual accountability, but this could be 
remedied by assigning research and publishing tasks to individuals. 

As ESSC exists, it seems to be tailored to those in a formal study. A teacher using the web may 
stumble across it but find it minimally useful due to the lack of teacher friendly materials such as 
answer keys, and objectives, and the general information overload in each area. If program 
participants receive more information than what is presented on this site, the program has great 
potential to revolutionize the way students learn in schools. They would use data, interact, and 
think critically about information that has a profound affect on our world. 

Based on the university science pre-service teacher educator review, the following 
recommendations are made regarding this site: 

Based on the teacher review, the following recommendations are made regarding this site: 

1 . Rework parts of this site so that it would be useful to teachers not able to integrate the whole 
concept into the curriculum. It may be possible to make some of the material more 
appropriate for younger students as well. 

For example, The Visual Data Browser could be used in 4 lh -6 lh grade in a math and/or science 
classroom. Students could compare data of our temperatures over the years. Students, who 
continue to investigate the uses of a line graph, could interpret comparative years. If solely using 
this portion of the site, a teacher could download links on a zip disk, or print out pages for the 
students to use. This is the portion of the site that could be further developed for a greater age 
range. If it were done, a separate link on ESSC homepage should be developed for “Lessons for 
Teachers on Global Warming.” It could be interesting to have students come up with conclusions 
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using the data, and then compare it to scientists’ conclusions. To continue the process, students 
could record temperatures of their area for the year, and compare it to former records. 


University Level Instructional Design Perspective 

The projects that are described in the curriculum section of ESSC are among the most innovative 
and potentially engaging for high school students. The Earth System projects, for example, 
include a series of interesting questions, and the use of tools, such as using spreadsheet to plot 
monthly averaged temperature data of the past thirty years and calculating averages. 

Visualization tools, such as Spyglass and Excel, and dynamic modeling tools, such as STELLA, 
are used to make simple models of the Earth's energy input and output. Students also use remote 
sensing to collect observations. Finally, students generate web pages to describe their research 
and exercises. This is a powerful and complete combination of technology-supported learning 
activities. 

The questions (such as those in the Fundamentals sections of the Physical Climate and 
Hydrologic Cycle sections) are interesting and should be stimulating. Clearly, not all of the 
information needed to answer these questions is provided at this site. That is acceptable if web 
searching is taught as a companion activity. 

The tool-oriented approach is admirable. The Earth System Visualizer is a powerful visualization 
tool that enables the user to examine cloud cover, ozone, temperature, reflectance, etc. on the 
earth's surface and plot that data or produce a color map for any point on the earth. The multiple 
views (graph vs. Colormap) enables learners adapt the presentation to their own preference. 

The best part of the ESSC site is the student reports. What ESSC represents is an extremely 
forward-thinking science program. The ideas, goals, objectives, etc. are great. They represent the 
state-of-the-art in science education. In order to make this more useful, more specific help is 
needed on-site for both teachers and students. 

Based on the instructional design review, the following recommendations are made regarding this 
site: 

1. Some of the activities (e.g. "Next, organize a list of questions that would have to be answered 
in order to address your hypothesis/research question and present preliminary report on your 
hypothesis/research question and outline your research strategy - same for Hydrologic Cycle) 
are vague. Such projects would require a good deal of support. The projects under the 
Biogeochemical section (e.g. Background Research, Hypothesis, Qualitative Modeling, and 
Preliminary Reports need a lot of specification. They presume a high level of pre-existing 
skills and a host of self-regulation strategies. 

2. It is evident from the Student Reports section that students were successful in this curriculum. 
Obviously, they had a great deal of help from local sources. But what is not clear is how the 
learners progressed or performed the projects in the curriculum. Add missing information to 
clarify. 

3. Add more specific advice to the Instructional Strategies section to help teachers figure out 
"how did they do that?" 
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4. Many more "help" links could be embedded in this site to help learners to perform these 
tasks. Tie the project activities in the project descriptions in the curriculum section to 
specific strategies, activities, and timelines and tie those to the student results. These 
activities would provide coaching, modeling of the activities, or some scaffolding to help 
learners do this. For instance, learning how to create Stella models, especially for high 
school students, is very challenging. Teachers would need to know how to do this. 

5. Within the overview, state the optimal background knowledge that a teacher and student need 
to use the program. 

6. Add answers to the kinds of questions that a prospective teacher may have: A teacher that is 
considering the program may be interested in the problems with the program, or the possible 
benefits. How has it fit into a schools curriculum? Have other schools used parts of the 
program, instead of the full year study? 

7. Add a “teacher talk” section to the overview, where teachers have a place to comment and 
others may question. 

8. If there is an intention is to solicit new teachers, the overview link should be the first. 

9. Reorganize each “project” page so that it is clear what subsequent pages contain objectives. 
Objectives are found in the overview for one project, in the schedule in another. Include 
NEXT buttons for sections that have a linear relationship, rather than forcing the user to use 
the browser’s BACK feature to return to previous page and make another selection. 

10. Add assignments into the timetables. 

11. Add a section to the assignments and quizzes section with not only with the answers, but also 
with specific objectives and how they are achieved. There is a general statement about 
philosophy that gives a theoretical overview, but there is a need for concrete statements to 
follow. 

12. A link to print out a manual with the timetables, expectations of teachers and students, 
objectives, assignments, quizzes etc. would be a valuable organizational resource for 
teachers. 
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CONCLUSIONS 


Based on this review, the following conclusions are drawn regarding the “school readiness” of the 

sites. 

• All six of the sites had great appeal to the reviewers and therefore we can conclude that each 
has tremendous potential for use in the classroom. 

• None, however, were given a completely “school-ready” assessment. With recommended 
revisions, each site will be a valuable asset to the classroom teacher. 

• For each as well, teachers particularly had problems with access or finding them, but once 
there, felt they all had some relevance. 

• Many of the weaknesses noted related to lack of guidance for teachers with links to their 
curriculum or provision of activities. 

• Other weaknesses related to the lack of educational purpose, however, this may be addressed 
in the development of strong teacher resources and guides linked to the sites. 

• For those sites with a weak or without a teacher’s guide, adding this component would be 
worth the expenditure of time and money needed. 
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APPENDIX A— DESCRIPTION OF TEAM MEMBERS 


K-12 Classroom Teacher Team 

The following six teachers reviewed the site. 

1. Thomas Arnold, Ph.D. - Tom Arnold has been teaching in the public schools for 27 years, 
and in the State College School District for 24 of those years. Tom is currently teaching 
courses in the introductory earth sciences and the advanced earth science electives. 

2. Beth Sockman - Beth is a 6 th grade science teacher at Stroudsburg Middle School in 
Northeast, PA. 

3. Steve Lindberg - Steve teaches Earth Science at Westmont Hilltop High School in Johnstown 
Pennsylvania. He currently serves as the National Association of Geoscience Teachers state 
councilor for Pennsylvania. 

4. Ellen Kendall - Ellen has an extensive teaching background including everything from 
English and Social Studies to Computer Programming. She now teaches courses in 
multimedia and software development at the Center for Advanced Technologies in Altoona, 
PA. 

5. John Hopkins - John teaches physics and physical science related courses. John has given 
presentations concerning the use of computer applications in the classroom. In addition, he 
has been an advisor to ISEE Corporation for developing interactive astronomy software. 

6. William Stacy - Bill has 25 years teaching experience having taught 8 years of middle school 
science and 17 years of high school physics and biology. He currently teaches at Eastern 
Technical High School in Baltimore County, Maryland. 


University Science Pre-Service Teacher Educators 

Dr. Vince Lunetta led the science education perspective. Formerly a physics/physical science 
teacher and department head, Dr. Lunetta was on the faculty at the University of Iowa. He also 
had a position with the National Science Foundation prior to joining the faculty as a full professor 
at the Pennsylvania State University, 

Susan J. Feeley, an advanced Ph.D. candidate at Penn State, provided the math education 
viewpoint. She currently works with Dr. Kathleen Heid, who has managed many NASA 
sponsored fellows at Penn State University. 


University Level Instructional Designers 

Professor David Jonassen of Pennsylvania State University led the instructional systems design 
point of view. Dr. Jonassen is Professor of Instructional Systems in the Department of Adult 
Education, Instructional Systems, and Workforce Education and is a very active researcher and 
author. His research focuses on constructivistic learning environments, and the use of technology 
to support those environments. 
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Summaries of all points of view from an instructional systems design perspective was provided 
by Jim Lloyd, project coordinator, and graduate student in the Instructional Systems Program at 
Penn State. 

Dr. Barbara Grabowski, Associate Professor in the Instructional Systems Design Program and 
Principal Investigator to the project provided project oversight. She has over twenty-five years of 
experience in education from being an elementary education teacher to designer/manager on large 
instructional design projects prior to coming to Penn State. She is a former NASA fellowship 
recipient. 
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